An Average Curvature Accelerated Composite Gradient Method for Nonconvex Smooth Composite Optimization Problems
نویسندگان
چکیده
This paper presents an accelerated composite gradient (ACG) variant, referred to as the AC-ACG method, for solving nonconvex smooth minimization problems. As opposed well-known ACG variants that are based on either a known Lipschitz constant or sequence of maximum observed curvatures, current one is average all past curvatures. More specifically, uses positive multiple curvatures until previous iteration way estimate “function curvature” at point and then two resolvent evaluations compute next iterate. In contrast other variable estimation variants, e.g., ones curvature, always accepts aforementioned iterate regardless how poor turns out be. Finally, computational results presented illustrate efficiency both randomly generated real-world problem instances.
منابع مشابه
Gradient Descent with Proximal Average for Nonconvex and Composite Regularization
Sparse modeling has been highly successful in many realworld applications. While a lot of interests have been on convex regularization, recent studies show that nonconvex regularizers can outperform their convex counterparts in many situations. However, the resulting nonconvex optimization problems are often challenging, especially for composite regularizers such as the nonconvex overlapping gr...
متن کاملAccelerated Stochastic Gradient Method for Composite Regularization
Regularized risk minimization often involves nonsmooth optimization. This can be particularly challenging when the regularizer is a sum of simpler regularizers, as in the overlapping group lasso. Very recently, this is alleviated by using the proximal average, in which an implicitly nonsmooth function is employed to approximate the composite regularizer. In this paper, we propose a novel extens...
متن کاملAn Asynchronous Distributed Proximal Gradient Method for Composite Convex Optimization
xi=x̄i when ‖∇xif(x̄)‖2 ≤ λBi, it follows that x̄i = x̄i if and only if ‖∇xif(x̄)‖2 ≤ λBi. Hence, hi(x̄ ∗ i ) = 0. Case 2: Suppose that i ∈ Ic := N \ I, i.e., ‖∇xif(x̄)‖2 > λBi. In this case, x̄i 6= x̄i. From the first-order optimality condition, we have ∇xif(x̄) + Li(x̄i − x̄i) + λBi x̄ ∗ i −x̄i ‖x̄i −x̄i‖2 = 0. Let si := x̄∗i −x̄i ‖x̄i −x̄i‖2 and ti := ‖x̄i − x̄i‖2, then si = −∇xif(x̄) Liti+λBi . Since ‖si‖2 = 1, i...
متن کاملAn Asynchronous Distributed Proximal Gradient Method for Composite Convex Optimization
We propose a distributed first-order augmented Lagrangian (DFAL) algorithm to minimize the sum of composite convex functions, where each term in the sum is a private cost function belonging to a node, and only nodes connected by an edge can directly communicate with each other. This optimization model abstracts a number of applications in distributed sensing and machine learning. We show that a...
متن کاملGradient Sliding for Composite Optimization
We consider in this paper a class of composite optimization problems whose objective function is given by the summation of a general smooth and nonsmooth component, together with a relatively simple nonsmooth term. We present a new class of first-order methods, namely the gradient sliding algorithms, which can skip the computation of the gradient for the smooth component from time to time. As a...
متن کاملذخیره در منابع من
با ذخیره ی این منبع در منابع من، دسترسی به آن را برای استفاده های بعدی آسان تر کنید
ژورنال
عنوان ژورنال: Siam Journal on Optimization
سال: 2021
ISSN: ['1095-7189', '1052-6234']
DOI: https://doi.org/10.1137/19m1294277